1. Field of the Invention
The present invention relates to a display method for a multimedia mobile terminal, and more particularly to an intelligent display method for a multimedia mobile terminal, wherein, when the mobile terminal displays a sport game video, the entire frame is displayed in the case of a non-long-distance shot, but, in the case of a long-distance shot, in which objects appear smaller, only the region of interest existing within the shot is magnified and displayed.
2. Description of the Related Art
An increasing number of people watch videos via small LCD panels as a result of the rapid development of multimedia signal processing and transmission technologies, as well as the appearance of new types of mobile TV services, including DVB-H (Digital Video Broadcasting-Handheld) and DMB (Digital Multimedia Broadcasting). However, most service providers simply reduce conventional images and provide them as mobile broadcasts primarily for cost-related reasons.
Experiments conducted by Knoche et al. to determine conditions (pixel number, bit rate, etc.) suitable for displaying reduced images via mobile terminals are described in a paper by H. Knoche, J. D. McCarthy, and M. A. Sasse, entitled “Can small be beautiful?: assessing image resolution requirements for mobile TV”, in MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia, pp. 829-838, ACM Press (New York, N.Y., USA), 2005. The experimental results show that, when images are directly reduced for display via compact terminals, visual details are lost. Such a loss is more severe when images are related to field sports, particularly in the case of a soccer video. The loss lessens little by little in the order of music videos, news, and animation videos.
One solution to this problem of loss of detail is to develop intelligent display technology so that, in the case of a video containing specific contents, only the region of interest (hereinafter, referred to as ROI) is magnified and displayed to the user of a compact mobile terminal via the screen. As used herein, the ROI refers to a region of the screen, in which users are most interested, or to which users pay more attention than other regions. The ROI is used for situation recognition content adaptation, transcoding, intelligent information management, and the like. In addition, designation of the ROI may be the first step for analyzing video scenes in terms of their meaning. Therefore, such technology is very important also with regard to image analysis.
FIG. 1 illustrates three types of shots existing in a conventional sport game video. In the drawing, (a) refers to a long-distance shot, (b) refers to a medium-distance shot, and (c) refers to a short-distance shot. In the case of the long-distance shot (a), the ROI needs to be extracted for magnification and playback.
Various methods have been studied to determine the ROI. For example, a paper by L. Itti, C. Koch, and E. Niebur, entitled “A model of saliency-based visual attention for rapid scene analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, November 1998, proposes that, in order to find visually noticeable portions within images, a number of spatial/visual saliencies are expressed by a single saliency map based on a visual attention model. However, extension to moving images has not been considered properly in this model.
In addition, ROI determination in videos based on information regarding brightness, color, and motion is disclosed in a paper by W. H. Cheng, W. T. Chu, and J. L. Wu, entitled “A visual attention based region-of-interest determination framework for video sequences”, IEEE Transactions on Information and Systems, E-88D, pp. 15781586, 2005. Based on an assumption that important objects have a high contrast of color or brightness, the researchers propose a method for designating the most salient point in each image frame. However, the above assumption is not always valid, because important objects may be dark or have a low contrast.
For the sake of video skimming and summarization, a paper by Y. F. Ma and H. J. Zhang, entitled “A model of motion attention for video skimming”, in Proc. ICIP, pp. 129-132, 2002, proposes a video analysis method for providing a user-interested model by using motions, speeches, camera works, video editing information, etc.
Although the above-mentioned methods can be used to extract ROIs based on saliencies within video screens, they are not suitable for intelligent display. More particularly, in the case of a field sport video (e.g. soccer game video), shots requiring ROI extraction are mingled with those requiring no ROI extraction. In the former case, a number of small objects having saliencies may exist on the screen simultaneously. This means that a plurality of ROIs may exist, and such a situation is not suitable for an intelligent display method for mobile terminals, which aims at extracting only a portion from the screen for magnification and display.
In an attempt to solve these problems, an application P2006-28802 filed in the Korean Intellectual Property Office by KIM, Chang-Ik, et al., May 2006, entitled “An Intelligent Soccer Video Display Scheme for Mobile Devices”, provides a display method including three steps: a ground color learning step, a shot classification step, and an ROI determination step.
However, the provided display method has a problem in that the processing time is lengthened by the first step (i.e. ground color learning step) during an initial period when each video starts. In addition, when the stadium currently displayed on the screen is replaced with another while a soccer game is broadcasted, it is difficult to properly adapt to the new ground color. During the third step for automatically determining the ROI, the ball is searched out for each frame. This means that the entire screen is searched for every frame. As a result, the processing rate cannot be increased further.